Skip to content

perf(monitor): probe the cached blob instead of walking every 10 s - #46

Closed
Lyrex wants to merge 10 commits into
WyrmStudios:mainfrom
Lyrex:probe-monitor
Closed

Lyrex wants to merge 10 commits into
WyrmStudios:mainfrom
Lyrex:probe-monitor

Conversation

@Lyrex

@Lyrex Lyrex commented Aug 9, 2026

Copy link
Copy Markdown
Contributor

perf(monitor): probe the cached blob instead of walking every 10 s

Depends on #42. This branch is #42 plus one commit, so it will not apply cleanly
until that merges.

The problem. The monitor calls capture_all_blobs every 10 s. That function
walks every committed region in the game process, reads gigabytes, and drops the
player's framerate for seconds at a time. It runs on that fixed cadence whether or
not the inventory changed, and the inventory changes maybe once per mission.

capture_all_blobs already had a fast path for this: probe the address the last
successful scan found the blob at, and if it is still there, skip the walk. But the
fast path lives inside the walk function, so a miss means the walk runs anyway.
At a 10 s cadence with a stale address that is a full walk every 10 s for as long
as it stays stale.

What lands. One commit, three pieces.

  1. probe_tick, the fast path pulled out into its own function, with no walk
    behind it. Returns ScanOutcome::{Updated, Unchanged, CacheMiss} and never
    escalates on its own. The monitor polls it every 2 s; a hit costs a few
    megabytes of reads.
  2. walk_is_due, the escalation policy, pure and unit-tested. Updated
    never walks, since the probe already delivered the inventory. CacheMiss with a
    sync marker walks immediately. CacheMiss on a known blob keeps the old 10 s
    cadence. A client with no blob yet, meaning the login screen, waits 60 s. A
    settled inventory answering Unchanged waits 900 s, which covers the one case
    the probe genuinely cannot see: the game reallocated the blob but the old
    address still holds a parseable copy of the old bytes.
  3. The inventory-sync marker, read out of process memory. Warframe logs
    OnInventoryResults completed in 339ms when it finishes fetching inventory.
    That line is the signal that a walk is worth doing. The obvious source is the
    EE.log tail you already run, and this adds that, one buf.contains in
    start_log_watcher. But the game buffers its log writes and flushes in
    bursts. Sampling a live client showed the newest in-memory log line running
    23 s ahead of the newest line on disk, so the tail alone would put that
    delay in front of every capture. probe_tick therefore also finds the line by
    content in the game's own log buffers, identified by Sys [Info]: preceded by
    a seconds-since-launch timestamp. No pointer chain, no per-build offsets; the
    buffer address caches like LAST_BLOB_REGION does, with a 30-probe backoff if
    it cannot be found at all.

What this changes in behaviour. In the common case, playing with the inventory
settled, the walk stops happening and the 2 s probe replaces it. When the
inventory does change, capture is faster than before, because the marker
escalates immediately instead of waiting out the remainder of the 10 s slot.

The premise I could not verify on your platform. The 23 s figure and the claim
that the marker is findable by content in the Windows client's memory both come
from my Linux fork against a Proton-hosted client, sampled with process_vm_readv.
The Windows arm here is re-authored against ReadProcessMemory / VirtualQueryEx
and has never been run. If the buffers are laid out differently on Windows and
the cold search never finds them, LOG_SEARCH_BACKOFF sits it out and the EE.log
tail carries the marker instead, so the degradation is latency, not breakage. But
you should expect to check that specifically.

Verification.

  • memory_scanner.rs: the scanner tests run on Linux (26 pass, 7 of them new
    around the marker parsing and the steady-state notice) and the Windows arm
    type-checks under --target x86_64-pc-windows-gnu.
  • walk_is_due: 7 tests, one per rule. They are in lib.rs, which does not
    compile here, so I ran them by extracting the function and its tests into a
    standalone file. They pass. The function is pure, four arguments in and a bool
    out, so that is a real check of the policy, but it is not a check that the
    surrounding lib.rs compiles.
  • lib.rs is not compiled. No MSVC linker, and --target x86_64-pc-windows-gnu
    dies in ring's build script for want of mingw. Expect to fix something.
  • Never run against a live Warframe process on Windows.

If you reimplement this rather than taking the commit, the load-bearing parts are:

  1. The probe must not fall back to a walk. That is the entire point of
    splitting it out of capture_all_blobs. A probe that walks on miss, polled at
    2 s, is strictly worse than the 10 s cadence it replaced.
  2. capture_all_blobs must lose its inline fast path. The monitor already ran
    that scan and escalated on its result; running it again at the top of the walk
    answers the same and skips the walk that was just asked for.
  3. Every rule in walk_is_due needs an interval that fires without a marker.
    The marker is an optimisation. If any branch depends on it for correctness, a
    client with no EE.log and unlocatable log buffers stops updating entirely.
  4. ScanOutcome::Unchanged plus a sync marker must still walk. "The client
    fetched inventory and our copy did not move" is indistinguishable from "our
    address is stale and still holds the old bytes". The probe cannot tell them
    apart, so the marker has to win.
  5. The marker flag must be held, not read live. The memory probe reports each
    marker exactly once. A marker that arrives on a tick which cannot act on it
    (rate limit, walk in flight) is lost unless it is folded into a sticky flag and
    cleared only when acted on.
  6. Clear the marker baseline on PID change, and count the first observation.
    The timestamps are seconds since that client launched, so a baseline from the
    previous process swallows everything the new one writes. And the login sync,
    the one that populates the inventory, arrives moments after the PID change, so
    spending the first observation on establishing a baseline drops it on every
    restart.
  7. Take the newest marker in the buffer, not the last one. The heap ring wraps,
    so the newest line is not at the highest address.
  8. Reject marker matches with no timestamp in front of them. The format string
    OnInventoryResults completed in %dms lives in .rdata and matches the same
    bytes. Skipping executable pages catches most of it; the timestamp check catches
    the rest.
  9. Back off the cold search after it fails. It reads every non-executable
    region under 16 MB. Without a backoff, a client whose buffers cannot be located
    pays that on the monitor thread every 2 s for the whole session.

Lyrex added 10 commits August 7, 2026 19:52
- `scan_completion_date_context`: `#[allow(dead_code)]`, no callers.
- A verbatim duplicate of the first line of the `ArchonShard` doc
  comment.
- A truncated doc fragment above the full-account blob parser section
  header; it repeats the opening of the `capture_all_blobs` doc block,
  which still sits intact on that function further down.
- `capture_inventory_blob`, its `#[tauri::command]` wrapper and the
  invoke_handler registration: no frontend call site, so the command
  was unreachable.
`scan_auth_credentials` and `scan_steam_id` ran byte-at-a-time loops
that scanned for a single anchor byte (`"`, `a`, `s`) and then compared
the following slice against the full needle. These functions run per
memory region over multi-gigabyte address space scans, so every
anchor-byte false positive paid for a full needle comparison.

Replace each loop with `memchr::memmem` (SIMD substring search).
Match positions and extraction bounds are unchanged.

One deliberate narrowing: the old loops resumed at `next + 1` and so
could match overlapping occurrences of a needle, while `find_iter`
resumes past the end of a match. Of the five needles only `"id":"`
has a self-border, and exercising the difference requires a buffer
holding the literal `"id":"id":"` followed by a valid account id. The
game's JSON cannot produce that, so the change is not observable.

memchr is already in Cargo.lock as a transitive dependency of
aho-corasick and regex, so depending on it directly leaves the
dependency tree and the resolved version untouched.
find_blob_end, blob_seed_offsets, and extract_blob_json searched
multi-megabyte inventory blobs with scalar `windows().position()`
loops. Swap these for memchr::memmem::find, which is SIMD-accelerated,
and use memchr::memchr for the single-byte closing-brace scan in
find_blob_end.

extract_blob_json's start-marker re-scan now reuses the existing
START_MARKER constant instead of a duplicate local one, since both
searched for the same bytes.

enclosing_object_start stays a manual backward loop: it counts brace
depth to find the *enclosing* `{`, not the last occurrence of a byte,
so memrchr does not apply.

Two of the converted sites were unpinned: the `.or_else` marker
priority (the existing assertion held for either branch) and the
extract_blob_json path that reinstates a missing opening brace. Both
now have exact-offset assertions, so a drift in search order or offset
arithmetic fails the harness.
parse_full_account_blob aggregated Upgrades/RawUpgrades/FlavourItems/
WeaponSkins into hash maps via entry(it.to_string()), which allocates
a String on every one of thousands of entries even though these
sections are dominated by duplicate keys. Check get_mut first and
only allocate on first insert.
Each ~10s scan cycle rebuilds BlobInventory's HashMaps/Vecs from a
multi-megabyte JSON tree, even though inventory changes maybe once
per mission, so almost every parse was wasted.

Add LAST_BLOB_DIGEST beside the existing LAST_BLOB_REGION address
cache, and blob_unchanged() to compare a candidate blob's bytes
against the last one seen. Check-and-update in a single call keeps
each of the three call sites (fast-path re-probe, full-walk
completion, full-walk immediate hit) one early-exit instead of
separate check and update steps. The digest is recorded before the
parse outcome is known, so every parse-failure path drops the
baseline via forget_blob_digest(); otherwise persistent unparseable
bytes would read as a cached success and stall the walk until the
PID changed.

On a hit, the scan is dropped without parsing or sending to blob_tx.
The UI already tolerates a cycle producing nothing and keeps its
last-known state, so the "Inventory Loaded" badge now appears when
the inventory actually changed rather than every 10s. The skip is
gated on `!save`, matching the existing fast-path convention, so an
explicit debug dump still writes the file even when the blob hasn't
moved. reset_last_blob_region() also clears the digest so a PID
change forces a re-parse.
extract_blob_json always copied the blob region into a fresh Vec,
even in the common case where the buffer still starts with the
original `{` and the bytes could be returned as a slice. For
multi-megabyte scan buffers this is a needless memcpy on every parse.

Add extract_blob_json_ref, returning Cow<[u8]>: Cow::Borrowed when
the opening brace survived, Cow::Owned only in the fallback path
where it was overwritten and has to be reinstated.
extract_blob_json is now a thin extract_blob_json_ref(..).into_owned()
wrapper, so its signature and behavior are unchanged for existing
callers.

parse_full_account_blob already computes end_pos via find_blob_end
for its size check, then extract_blob_json used to redo that scan
and re-locate the start marker. Factor the shared body into
extract_blob_json_at(raw, end_pos), taking the bound as a parameter,
and have parse_full_account_blob call it directly with the end_pos
it already has.
`blob_unchanged` hashed the whole buffer it was handed, and all three
callers hand it a stitched one. The stitch stops at the region that closes
the JSON but keeps everything after the closing brace, which is whatever
heap shares that region: tens of megabytes of it, rewritten constantly by
a running client.

The digest therefore changed on every scan of a completely settled
inventory, so the skip never fired once and every cycle reparsed. Hashing
now stops at `find_blob_end`, which also means hashing a few MB rather
than the whole read. Buffers with no blob end are hashed whole; they are
not blob data and only need to compare equal to themselves.

The existing fixtures could not catch this. Both carry no `DeathSquadable`
key and no trailing bytes, so they exercise only the whole-buffer path.

The digest tests also reset a process-global static and then assert
against it while running in parallel, so they could already clobber each
other. They share a mutex now.
The blob qualification runs six needles over every committed region, and
the end-marker check runs over the whole pending scan buffer. Those were
still scalar `windows().any()` passes while the rest of the scanner had
moved to `memmem`, so they fell back to a byte-at-a-time compare on
buffers measured in tens of megabytes.

`memmem::find(..).is_some()` matches the old semantics on short slices
too: a haystack shorter than the needle yields no windows and no match.
A freed copy of the FULL_ACCOUNT blob can sit ahead of the live one
with its start marker intact but its opening brace overwritten.
Matching braces backward from that copy walks into binary garbage and
seeds at a stray `{`, so the parse dies ("key must be a string at line
1 column 2") and the scan drops the inventory even though a healthy
copy is in the same buffer.

Accept a candidate brace only when a quote follows it, the shape of a
real JSON object head, and keep trying later marker occurrences until
one qualifies. When none does, seed at the first marker itself and let
the parser rebuild the object head, so a lone headless copy still
yields its inventory.

The marker walk uses `memmem::find` to match the scans earlier in this
branch.
The monitor walked every committed region of the game process every 10
seconds, reading gigabytes to find a blob that usually had not moved
since the last pass. That walk costs the player frames for seconds at a
time, and it ran whether or not the inventory had changed.

Split the cached-address fast path out of capture_all_blobs into
probe_tick, which reads the remembered address and nothing else, and
never falls back to a walk of its own. The monitor polls it every 2 s and
decides for itself when a result is worth escalating: a fresh parse needs
no walk at all, a cache miss with a sync marker walks immediately, and
everything else is rate-limited by walk_is_due.

The marker itself comes from two places. Warframe formats its log lines
in memory well before flushing them to EE.log, and sampling a live client
showed the in-memory copy running 23 s ahead, so probe_tick finds the
newest "OnInventoryResults completed in" line by content in the game's
log buffers, and the existing EE.log tail sets the same flag whenever it
gets there first. Neither is required: every case in walk_is_due has an
interval that fires without a marker, so a missing EE.log or an
unlocatable buffer costs latency only.
@Lyrex

Lyrex commented Aug 10, 2026

Copy link
Copy Markdown
Contributor Author

thank you for integrating the PRs into the main repo! verified it on/against the linux branch, and everything is working well!

@Lyrex Lyrex closed this Aug 10, 2026
@Lyrex
Lyrex deleted the probe-monitor branch August 10, 2026 20:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant